Methods and apparatus to provide machine assisted programming

ABSTRACT

Methods, apparatus, systems and articles of manufacture to provide machine assisted programming are disclosed. An example apparatus includes processor circuitry to execute computer readable instructions to: execute a machine learning model to generate a first code recommendation for programming code, the first code recommendation being associated with security of the programming code; cause output of the first code recommendation via a user interface; update the machine learning model based on feedback obtained via the user interface; determine a performance of the programming code; generate a second code recommendation, the second code recommendation being associated with the performance of the programming code; and cause output of the second code recommendation via the user interface.

RELATED APPLICATION(S

This patent arises from a continuation of U.S. Pat. Application No.17/967,830 (now U.S. Pat. ______) which was filed on Oct. 17, 2022,which is a continuation of U.S. Pat. Application No. 16/457,365 (nowU.S. Pat. 11,475,369) which was filed on Jun. 28, 2019. U.S. Pat.Application No. 17/967,830 and U.S. Pat. Application No. 16/457,365 arehereby incorporated herein by reference in their entireties. Priority toU.S. Pat. Application No. 17/967,830 and U.S. Pat. Application No.16/457,365 is hereby claimed.

FIELD OF THE DISCLOSURE

This disclosure relates generally to machine learning, and, moreparticularly, to methods and apparatus to provide machine assistedprogramming.

BACKGROUND

In recent years, machine learning and/or artificial intelligence haveincreased in popularity. Machine learning includes the use of models,algorithms, and/or computer systems to perform specific tasks withoutexplicit instructions. A machine learning model can be trained (e.g.,can learn) to perform the specific tasks based on feedback so that theoutput corresponds a desired result. Once trained, the machine learningmodel can make decisions to generate an output based on any input.Machine learning models are used for the emerging fields of artificialintelligence as well as other technologies.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example computer device providingmachine assisted programming.

FIG. 2 is a flowchart representative of example machine readableinstructions which may be executed to implement the example computerdevice of FIG. 1 to train the example machine programming solver of FIG.1 .

FIGS. 3A-3B is a flowchart representative of example machine readableinstructions which may be executed to implement the example computerdevice of FIG. 1 to provide machine assisted programming.

FIG. 4 is a block diagram of an example processing platform structuredto execute the instructions of FIGS. 2, 3A, and/or 3B to implement theexample computer device of FIG. 1 .

The figures are not to scale. In general, the same reference numberswill be used throughout the drawing(s) and accompanying writtendescription to refer to the same or like parts. Connection references(e.g., attached, coupled, connected, and joined) are to be construedbroadly and may include intermediate members between a collection ofelements and relative movement between elements unless otherwiseindicated. As such, connection references do not necessarily infer thattwo elements are directly connected and in fixed relation to each other.Although the figures show layers and regions with clean lines andboundaries, some or all of these lines and/or boundaries may beidealized. In reality, the boundaries and/or lines may be unobservable,blended, and/or irregular.

Descriptors “first,” “second,” “third,” etc. are used herein whenidentifying multiple elements or components which may be referred toseparately. Unless otherwise specified or understood based on theircontext of use, such descriptors are not intended to impute any meaningof priority, physical order or arrangement in a list, or ordering intime but are merely used as labels for referring to multiple elements orcomponents separately for ease of understanding the disclosed examples.In some examples, the descriptor “first” may be used to refer to anelement in the detailed description, while the same element may bereferred to in a claim with a different descriptor such as “second” or“third.” In such instances, it should be understood that suchdescriptors are used merely for ease of referencing multiple elements orcomponents.

DETAILED DESCRIPTION

Automation has increased the productivity of human labor taking overrepetitive tasks and performing semi-supervised operations at largescale. The advent of unsupervised learning is executed with automatedsystems that can learn on their own, given interactions with theenvironment, through optimization towards a predefined reward function.Artificial intelligence may be applied to a variety of problems, frompattern recognition to prediction engines, recommendation systems anddata-based analytics empower professionals in science and engineering.

As developers generate new computer programming code, developers mayoverlook the time, space complexity, memory consumption, security, etc.for the algorithms that they develop. For example, a developer maydevelop a code that has unintentional impacts when the domain input sizefor the algorithm increases, which may cause program crashes due to heapmemory limits, program crashes due to stack overflows through a longseries of function recursion invocation, program responsiveness offreezing due to an inefficient time complex algorithm, etc.

Examples disclosed herein utilizes a machine programming solver toutilize machine learning models to process generated code, determine thealgorithms (e.g., methods, classes, sub-sections of code, etc.) of thecode, and generate recommendations to replace one or more of thealgorithms of the code with algorithms that are more efficient for aparticular parameter (e.g., speed, memory, resources, security, etc.)and/or better suited for the purpose of the new code. During training,examples disclosed herein utilize feature vectors (e.g., representativeof algorithms) from libraries (e.g., stored locally at the computingdevice or at an external computing device and/or server) and/or internalcode to generate clusters of feature vectors representative ofalgorithms, where each cluster corresponds to a similar functionality(e.g., a sorting cluster, a transmission cluster, a searching cluster,etc.). Although the feature vectors in a cluster serve the same purpose,each algorithm corresponding to a feature vector may have differentadvantages and disadvantages.

Once the clusters have been developed, examples disclosed hereinorganize each cluster according to the advantages of the algorithmscorresponding to feature vectors within a cluster. For example, thefeature vector of the cluster may include documentation that identifiesthe best use of the corresponding algorithm and/or the parameterscorresponding to the algorithm (e.g., how fast the algorithm is, theamount of processor resources needed to execute the algorithm, thememory required to execute the algorithm, the limits of the algorithm,the security level of the algorithm etc.). Additionally oralternatively, the parameters of an algorithm may be determinedanalytically (e.g., by processing the code of the algorithm) or bymeasuring the parameters while executing the algorithm. Once theparameters are known, examples disclosed herein organize feature vectorsof a cluster based on efficiency of the parameters of the featurevectors.

When new code is generated, examples disclosed herein break the code canbe broken into feature vectors and identify any clusters that match thefeature vectors (e.g., based on a similarity determination). If afeature vector of the new code matches a cluster, exampled disclosedherein generates a recommendation for the feature vector by finding analgorithm corresponding to a feature vector of the cluster that is moreefficient for a particular parameter (e.g., the parameter beingidentified by a user and/or being based on context data). Therecommendation is provided to the user to create more efficient codethat operates as the code’s other intended without less possibility ofprogram crashes or freezing.

Artificial intelligence (AI), including machine learning (ML), deeplearning (DL), and/or other artificial machine-driven logic, enablesmachines (e.g., computers, logic circuits, etc.) to use a model toprocess input data to generate an output based on patterns and/orassociations previously learned by the model via a training process. Forinstance, the model may be trained with data to recognize patternsand/or associations and follow such patterns and/or associations whenprocessing input data such that other input(s) result in output(s)consistent with the recognized patterns and/or associations.

Many different types of machine learning models and/or machine learningarchitectures exist. In examples disclosed herein, a cluster model and arecommender model are used. Using a cluster model enables the groupingof similar blocks of code and a recommender model enables generation ofrecommendations for a user. In general, machine learningmodels/architectures that are suitable to use in the example approachesdisclosed herein could be any type of AI, ML, neural network, deeplearning, etc.

In general, implementing a ML/AI system involves two phases, alearning/training phase and an inference phase. In the learning/trainingphase, a training algorithm is used to train a model to operate inaccordance with patterns and/or associations based on, for example,training data. In general, the model includes internal parameters thatguide how input data is transformed into output data, such as through aseries of nodes and connections within the model to transform input datainto output data. Additionally, hyperparameters are used as part of thetraining process to control how the learning is performed (e.g., alearning rate, a number of layers to be used in the machine learningmodel, etc.). Hyperparameters are defined to be training parameters thatare determined prior to initiating the training process.

Different types of training may be performed based on the type of ML/AImodel and/or the expected output. For example, supervised training usesinputs and corresponding expected (e.g., labeled) outputs to selectparameters (e.g., by iterating over combinations of select parameters)for the ML/AI model that reduce model error. As used herein, labellingrefers to an expected output of the machine learning model (e.g., aclassification, an expected output value, etc.) Alternatively,unsupervised training (e.g., used in deep learning, a subset of machinelearning, etc.) involves inferring patterns from inputs to selectparameters for the ML/AI model (e.g., without the benefit of expected(e.g., labeled) outputs).

In examples disclosed herein, ML/AI models are trained using internallibraries (e.g., libraries included with a locally installed program) orexternal libraries (e.g., located on the Internet, the cloud, a server,etc.), internal code and/or new code. However, any other trainingalgorithm may additionally or alternatively be used. In examplesdisclosed herein, training is performed periodically, aperiodically, orbased on a trigger. In examples disclosed herein, training is performedlocally (e.g., at the computer device) or externally (e.g., at a serveror another device at then the trained models are deployed to thecomputing device). Training is performed using hyperparameters thatcontrol how the learning is performed (e.g., a learning rate, a numberof layers to be used in the machine learning model, etc.).

Training is performed using training data. In examples disclosed herein,the training data originates from publicly available libraries that arelocated locally or remotely (e.g., on the Internet, at a server, on thecloud, etc.). Because supervised training is used, the training data islabeled. In some examples, the training data is pre-processed using, forexample, feature extractor to convert code into vectors representativeof some or all of the code.

Once training is complete, the model is deployed for use as anexecutable construct that processes an input and provides an outputbased on the network of nodes and connections defined in the model. Themodel may be stored locally or remotely. The model may then be executedby the machine learning models.

Once trained, the deployed model may be operated in an inference phaseto process data. In the inference phase, data to be analyzed (e.g., livedata) is input to the model, and the model executes to create an output.This inference phase can be thought of as the AI “thinking” to generatethe output based on what it learned from the training (e.g., byexecuting the model to apply the learned patterns and/or associations tothe live data). In some examples, input data undergoes pre-processingbefore being used as an input to the machine learning model. Moreover,in some examples, the output data may undergo post-processing after itis generated by the AI model to transform the output into a usefulresult (e.g., a display of data, an instruction to be executed by amachine, etc.).

In some examples, output of the deployed model may be captured andprovided as feedback. By analyzing the feedback, an accuracy of thedeployed model can be determined. If the feedback indicates that theaccuracy of the deployed model is less than a threshold or othercriterion, training of an updated model can be triggered using thefeedback and an updated training data set, hyperparameters, etc., togenerate an updated, deployed model.

FIG. 1 illustrates an example computing device 100 providing machineassisted programming. The example computing system 100 includes anexample training database 102 receiving example libraries 104 andexample internal code 106, an example feature extractor 108 receivingexample new code 110, and an example machine programming solver 112. Theexample machine programming solver 112 includes an example clustermachine learning modem (MLM) 114, an example recommender MLM 116,example storage 118, and an example function executor 120. Although theexample computing system 100 includes the example training database 102,the example feature extractor 108, and the example machine programmingsolver 112 in the same computing system 100, one or more of the exampletraining database 102, the example feature extractor 108, and/or theexample machine programming solver 112 may be located remotely from thecomputing system 100 and may communicate via a wired or wirelesscommunication.

The example training database 102 of FIG. 1 stores blocks of code (e.g.,methods, functions, algorithms, classes, etc.). The blocks of code maybe provided by the example libraries 104, internal code 106, and/orparts of new code that does not correspond to a cluster. The libraries104 may be libraries of code local to the processing device and/or maybe libraries found outside of the processing device (e.g., on theInternet, in the cloud, at a server, etc.). The internal code 106 may beinternal repositories (e.g., git repositories) with optimizedimplementation, internal documentation (e.g., apache libraries), etc.stored within the processing device. The blocks of code may optimize oneor more particular aspects (e.g., speed, memory, resources, security,etc.) and/or are beneficial in particular contexts (e.g., code thatimplements technology best suited to be used in a particular context),as based on labels corresponding to the associated documentation todetermine the purpose or category of the code. Additionally, thetraining database 102 may store newly generated code (e.g., provided bythe machine programming solver 112) that does not correspond to acluster in the training database 102. In some examples, the blocks ofcode are stored in conjunction with functionality, performance data,and/or contextual data (e.g., if known). For example, the trainingdatabase 102 may include a block of code corresponding to a searchingfunction. Additionally, the block of code may be stored in the trainingdatabase 102 in conjunction with the amount of resources needed toperform the searching function, the amount of memory needed to performthe searching function, the speed and/or efficiency of performing thesearching function, the security of performing the searching function,etc. Such information may be provided by the creator of the block ofcode and/or a prior analysis of the block of code. The blocks of codestored in the example training database 102 are used to train theexample machine programming solver 112, as further described below. Theexample training database 102 may be located locally at the processingdevice and/or may be located remotely (e.g., in the cloud, at a server,etc.).

The example feature extractor 108 of FIG. 1 extracts features fromblocks of code in the training database 102 and/or based on newlywritten code 110. For example, periodically, aperiodically, and/or basedon a trigger (e.g., when new data has been added to the example trainingdatabase 102), the example feature extractor 108 may extract featuresfrom the block(s) of code stored in the example training database 102 toupdate the example cluster MLM 114 and/or the example recommender MLM116 based on new training data stored in the example training database102. The example feature extractor 108 uses the extracted features toconvert a block of code into one or more feature vectors representativeof the one or more features of the block of code. For example, a smallblock of code may be converted into a single feature vectorrepresentative of the block of code and a large block of code may beconverted into multiple feature vectors representative of the block ofcode as a whole, parts of the code (e.g., methods, classes, etc.),subparts of the code (e.g., one or more lines of code within a part ofthe code), etc. When the feature extractor 108 receives the example newcode 110 (e.g., code that has just been written and successfullycompiled without error), the example feature extractor 108 converts thenew code 110 into one or more features representative of the block ofcode as a whole, parts of the code, subparts of the code, etc. Theexample feature extractor 108 transmits the feature-based vectorrepresentation of blocks, part, and/or subparts of code to the examplemachine programming solver 112.

During training, the example cluster MLM 114 of FIG. 1 receives thefeature vectors (e.g., representative of a block of code of the newcode) from the example feature extractor 108 and includes the featureinto an established cluster of similar feature vectors (e.g.,representative of blocks of each of code from the training database 102)or generates a new cluster for new feature vectors that do notcorrespond to an established cluster. For example, the cluster MLM 114may generate a sorting cluster for feature vectors corresponding tosorting algorithms, a searching cluster for feature vectorscorresponding to searching algorithms, a transmission cluster forfeature vectors corresponding to transmission algorithms, etc. Duringtraining, the cluster MLM 114 learns the number of generated clusters,the centroids of the clusters, and/or a hierarchy of each cluster basedon the training data. The example cluster MLM 114 may be implementedusing k-means clustering, mean shift clustering, agglomerativehierarchical clustering, and/or any other type of clustering. Duringtraining, the cluster MLM 114 calculates training feature vectors andcluster assignments of the initial training data.

When the new code 110 is obtained and converted into feature vector(s),the example cluster MLM 114 of FIG. 1 associates the new featurevector(s) to a particular cluster(s) and passes the correspondingcluster(s) to the example recommender MLM 116. In this manner, theexample recommender MLM 116 can make recommendations to a user based onthe corresponding cluster. The example cluster MLM 114 may associate thenew feature vector(s) to particular cluster(s) by performing asimilarity measure between the feature vector(s) of the new code 110 andfeature vectors of the cluster. The similarity measure can be anypair-wise measure (e.g., a Euclidean distance, cosine similarity, etc.).If the example cluster MLM 114 determines that one or more of thefeature vectors of the new code 110 does not correspond to a cluster(e.g., the similarity measurement does not satisfy a threshold), thecluster MLM 114 transmits the feature to the example training database102 to be used to train the example cluster MLM 114 at a later point intime.

During training, the example recommender MLM 116 of FIG. 1 organizes thefeature vectors of a cluster according to the efficiency of one or moreparameters or context information. For example, the recommender MCM 616may generate a similarity matrix based on the clusters. The examplerecommender MLM 616 may be implemented using a content-based method, acollaborative filtering method, and/or any other known recommendersystem. In a content-based implementation, the example recommender MLM116 may maintain a similarity matrix for each cluster of the trainingdata calculated during training. The similarity matrix includesperformance and/or contextual ratings for each feature of a cluster. Inan item-based collaborative filtering implementation, the examplerecommender MLM 116 recommends optimized code blocks that are similar tothe current method or the suggest code blocks that were accepted byother programmers for similar code. If contextual and/or performanceinformation for a particular feature is not known (e.g., because it wasnot provided in documentation or provided from a user) during training,the example recommender MLM 116 may process the feature analytically todetermine the contextual and/or performance data. If contextual dataand/or performance data cannot be determined analytically, the examplefunction executor 120 may instruct the example function executor 120 toexecute the feature while the example recommender MLM 116 analyses theexecution of the function to determine contextual and/or performancedata corresponding to the feature.

When the example new code 110 is received, the example recommender MLM116 of FIG. 1 provides recommendations to the user for one or morealgorithms of the new code 110 based on the similarity matrix (e.g.,generated during training) corresponding to the identified cluster. Therecommendation is based on parameters to be optimized (e.g., speed,memory, resources, security, context, etc.). The parameters to beoptimized may be based on user preferences and/or context data, such ascurrent processor status (e.g., if the processor currently has limitedmemory, memory may become the parameter to be optimized or if theprogram manages sensitive data security may become the parameter to beoptimized). In some examples, the recommender MLM 116 generates arecommendation based on context data corresponding to an inter processcommunication mechanism (IPCM), input outputs (I/O), graphics,mathematical ops, security services, etc. IPCMs include local networkconnections (e.g., user datagram protocol (UDP)/ transmission controlprotocol (TCP) sockets), mailboxes, named pipes, shared memory, remoteprocedure calls (RPC) and/or other high level abstraction services forcommunication including different paradigms (e.g., RESTful, pub/sub,etc.), proxy services (e.g., d-bus), etc. Each of the IPCMs may bepreferential for different parameters (e.g., security, latency,bandwidth, complexity, robustness, portability, etc.). Accordingly, theexample recommender MLM 116 can make recommendations based on insightsas to the usage of the PCMs through runtime monitoring (e.g., number ofproducers, consumers, frequency of communication, data sizes, stream v.message based communication, etc.). The recommender MLM 116 may displaythe recommendation of the algorithm corresponding to the recommendedfeature vector to a user via a user interface in a ranked or unrankedorder. Additionally, the recommender MLM 116 may update organization ofrecommendation (e.g., the similarity matrix) based on feedback from theuser. For example, if a user selects one or more of the recommendations,the recommender MLM 116 can apply a larger weight to thoserecommendations for further recommendations.

In some examples, the example cluster MLM 114 and/or the examplerecommender MLM 116 are trained external from the example computersystem 100. For example, the cluster MLM 114 and/or the examplerecommender MLM 116 are trained on the cloud, at a server, and/or on anexternal device. In such examples, the trained cluster MLM and/or thetrained recommender MLM can be deployed to the example machineprogramming solver 112 via a wired or wireless communication.

The example storage 118 of FIG. 1 stores the functions, the functionclusters and/or hierarchy and the recommendation organization (e.g., thesimilarity matrix). When additional training occurs, the data stored inthe example storage 118 may be updated to reflect the updated functions,clusters, hierarchy, and/or similarity matrix.

While an example manner of implementing the computing system 100 isillustrated in FIG. 1 , one or more of the elements, processes and/ordevices illustrated in FIG. 1 may be combined, divided, re-arranged,omitted, eliminated and/or implemented in any other way. Further, theexample feature extractor 108, the example cluster MLM 114, the examplerecommender MLM 116, the example function executor 120, and/or, moregenerally, the example computing system 100 of FIG. 1 may be implementedby hardware, software, firmware and/or any combination of hardware,software and/or firmware. Thus, for example, any of the example featureextractor 108, the example cluster MLM 114, the example recommender MLM116, the example function executor 120, and/or, more generally, theexample computing system 100 of FIG. 1 could be implemented by one ormore analog or digital circuit(s), logic circuits, programmableprocessor(s), programmable controller(s), graphics processing unit(s)(GPU(s)), digital signal processor(s) (DSP(s)), application specificintegrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(s))and/or field programmable logic device(s) (FPLD(s)). When reading any ofthe apparatus or system claims of this patent to cover a purely softwareand/or firmware implementation, at least one of the example featureextractor 108, the example cluster MLM 114, the example recommender MLM116, the example function executor 120, and/or, more generally, theexample computing system 100 of FIG. 1 is/are hereby expressly definedto include a non-transitory computer readable storage device or storagedisk such as a memory, a digital versatile disk (DVD), a compact disk(CD), a Blu-ray disk, etc. including the software and/or firmware.Further still, the example computing system 100 of FIG. 1 may includeone or more elements, processes and/or devices in addition to, orinstead of, those illustrated in FIG. 1 , and/or may include more thanone of any or all of the illustrated elements, processes and devices. Asused herein, the phrase “in communication,” including variationsthereof, encompasses direct communication and/or indirect communicationthrough one or more intermediary components, and does not require directphysical (e.g., wired) communication and/or constant communication, butrather additionally includes selective communication at periodicintervals, scheduled intervals, aperiodic intervals, and/or one-timeevents.

Flowcharts representative of example hardware logic, machine readableinstructions, hardware implemented state machines, and/or anycombination thereof for implementing the example computing system 100are shown in FIGS. 2 and 3A-3B. The machine readable instructions may beone or more executable programs or portion(s) of an executable programfor execution by a computer processor such as the processor 412 shown inthe example processor platform 400 discussed below in connection withFIG. 4 . The program may be embodied in software stored on anon-transitory computer readable storage medium such as a CD-ROM, afloppy disk, a hard drive, a DVD, a Blu-ray disk, or a memory associatedwith the processor 412, but the entire program and/or parts thereofcould alternatively be executed by a device other than the processor 412and/or embodied in firmware or dedicated hardware. Further, although theexample program is described with reference to the flowchartsillustrated in FIGS. 2 and 3A-3B, many other methods of implementing theexample computing system 100 may alternatively be used. For example, theorder of execution of the blocks may be changed, and/or some of theblocks described may be changed, eliminated, or combined. Additionallyor alternatively, any or all of the blocks may be implemented by one ormore hardware circuits (e.g., discrete and/or integrated analog and/ordigital circuitry, an FPGA, an ASIC, a comparator, anoperational-amplifier (op-amp), a logic circuit, etc.) structured toperform the corresponding operation without executing software orfirmware.

The machine readable instructions described herein may be stored in oneor more of a compressed format, an encrypted format, a fragmentedformat, a compiled format, an executable format, a packaged format, etc.Machine readable instructions as described herein may be stored as data(e.g., portions of instructions, code, representations of code, etc.)that may be utilized to create, manufacture, and/or produce machineexecutable instructions. For example, the machine readable instructionsmay be fragmented and stored on one or more storage devices and/orcomputing devices (e.g., servers). The machine readable instructions mayrequire one or more of installation, modification, adaptation, updating,combining, supplementing, configuring, decryption, decompression,unpacking, distribution, reassignment, compilation, etc. in order tomake them directly readable, interpretable, and/or executable by acomputing device and/or other machine. For example, the machine readableinstructions may be stored in multiple parts, which are individuallycompressed, encrypted, and stored on separate computing devices, whereinthe parts when decrypted, decompressed, and combined form a set ofexecutable instructions that implement a program such as that describedherein.

In another example, the machine readable instructions may be stored in astate in which they may be read by a computer, but require addition of alibrary (e.g., a dynamic link library (DLL)), a software development kit(SDK), an application programming interface (API), etc. in order toexecute the instructions on a particular computing device or otherdevice. In another example, the machine readable instructions may needto be configured (e.g., settings stored, data input, network addressesrecorded, etc.) before the machine readable instructions and/or thecorresponding program(s) can be executed in whole or in part. Thus, thedisclosed machine readable instructions and/or corresponding program(s)are intended to encompass such machine readable instructions and/orprogram(s) regardless of the particular format or state of the machinereadable instructions and/or program(s) when stored or otherwise at restor in transit.

The machine readable instructions described herein can be represented byany past, present, or future instruction language, scripting language,programming language, etc. For example, the machine readableinstructions may be represented using any of the following languages: C,C++, Java, C#, Perl, Python, JavaScript, HyperText Markup Language(HTML), Structured Query Language (SQL), Swift, etc.

As mentioned above, the example processes of FIGS. 2 and 3A-3B may beimplemented using executable instructions (e.g., computer and/or machinereadable instructions) stored on a non-transitory computer and/ormachine readable medium such as a hard disk drive, a flash memory, aread-only memory, a compact disk, a digital versatile disk, a cache, arandom-access memory and/or any other storage device or storage disk inwhich information is stored for any duration (e.g., for extended timeperiods, permanently, for brief instances, for temporarily buffering,and/or for caching of the information). As used herein, the termnon-transitory computer readable medium is expressly defined to includeany type of computer readable storage device and/or storage disk and toexclude propagating signals and to exclude transmission media.

“Including” and “comprising” (and all forms and tenses thereof) are usedherein to be open ended terms. Thus, whenever a claim employs any formof “include” or “comprise” (e.g., comprises, includes, comprising,including, having, etc.) as a preamble or within a claim recitation ofany kind, it is to be understood that additional elements, terms, etc.may be present without falling outside the scope of the correspondingclaim or recitation. As used herein, when the phrase “at least” is usedas the transition term in, for example, a preamble of a claim, it isopen-ended in the same manner as the term “comprising” and “including”are open ended. The term “and/or” when used, for example, in a form suchas A, B, and/or C refers to any combination or subset of A, B, C such as(1) A alone, (2) B alone, (3) C alone, (4) A with B, (5) A with C, (6) Bwith C, and (7) A with B and with C. As used herein in the context ofdescribing structures, components, items, objects and/or things, thephrase “at least one of A and B” is intended to refer to implementationsincluding any of (1) at least one A, (2) at least one B, and (3) atleast one A and at least one B. Similarly, as used herein in the contextof describing structures, components, items, objects and/or things, thephrase “at least one of A or B” is intended to refer to implementationsincluding any of (1) at least one A, (2) at least one B, and (3) atleast one A and at least one B. As used herein in the context ofdescribing the performance or execution of processes, instructions,actions, activities and/or steps, the phrase “at least one of A and B”is intended to refer to implementations including any of (1) at leastone A, (2) at least one B, and (3) at least one A and at least one B.Similarly, as used herein in the context of describing the performanceor execution of processes, instructions, actions, activities and/orsteps, the phrase “at least one of A or B” is intended to refer toimplementations including any of (1) at least one A, (2) at least one B,and (3) at least one A and at least one B.

As used herein, singular references (e.g., “a”, “an”, “first”, “second”,etc.) do not exclude a plurality. The term “a” or “an” entity, as usedherein, refers to one or more of that entity. The terms “a” (or “an”),“one or more”, and “at least one” can be used interchangeably herein.Furthermore, although individually listed, a plurality of means,elements or method actions may be implemented by, e.g., a single unit orprocessor. Additionally, although individual features may be included indifferent examples or claims, these may possibly be combined, and theinclusion in different examples or claims does not imply that acombination of features is not feasible and/or advantageous.

FIG. 2 is an example flowchart 200 representative of example machinereadable instructions which may be executed to implement the examplecomputing system 100 of FIG. 1 to train the example cluster MLM 114 andthe example recommender MLM 116 to be able to provide machine assistantprogramming to a user.

At block 202, the example the example feature extractor 108 obtainstraining data for training. The training data may algorithms, methods,classes, and/or other blocks of code from stored in the example trainingdatabase 102. The training data may be from the example libraries 104(e.g., remote or local), the example internal code 106, and/or may bepart of the new code 110 that was flagged by the cluster MLM 114 as notbeing similar to any existing cluster.

At block 204, the example feature extractor 108 extracts feature(s) fromthe obtained training data. However, the example feature extractor 108may perform different types of transmissions for different featurerepresentations (e.g., to account for the model implementation). Thefeatures may represent an entire algorithm or part of an algorithm(e.g., a block, subblock, function, line, class, etc.). At block 206,the example generates a vector representation of the feature(s) toprovide to the example cluster MLM 114. For example, the featureextractor 108 may use a one-hot-encoding technique for categoricalfeature representations to generate one or more vector representationsof features of the obtained data. The vector representation allows forthe machine programming solver 112 to be able to interpret and/orprocess the data.

At block 208, the example cluster MLM 114 determines if there is one ormore of the feature vector(s) that does not correspond to a cluster. Forexample, the cluster MLM 114 may perform a similarity measure (e.g., aEuclidean distance, cosine similarity, etc.) between the featurevector(s) and the established clusters. If the example cluster MLM 114determines that there are not one or more feature vectors that do notcorrespond to a cluster (block 208: NO), control continues to block 212.If the example cluster MLM 114 determines that there are one or morefeature vectors that do not correspond to a cluster (block 208: YES),the example cluster MLM 114 generates a new cluster(s) for the one ormore feature vector(s) that do not correspond to a current cluster(block 210). The new cluster is stored in the example storage 118.

At block 212, the example cluster MLM 114 determines if there is one ormore of the feature vectors that corresponds to a cluster. If theexample cluster MLM 114 determines that there is not one or more featurevectors that corresponds to a cluster (block 208: NO), control continuesto block 216. If the example cluster MLM 114 determines that there isone or more feature vectors that do corresponds to a cluster (block 208:YES), the example cluster MLM 114 clusters the feature vectors into thecorresponding cluster(s) and/or cluster-based hierarchy(ies) of features(block 214). For example, if the cluster MLM 114 determines that one ofthe feature vectors corresponds to a sorting algorithm, the cluster MLM114 will include the feature with a cluster of feature vectorscorresponding to sorting algorithms. The example cluster MLM 114 storesthe updated and/or new clusters in the example storage 118 and providesthe updated and/or new clusters to the example recommender MLM 116.

At block 216, the example recommender MLM 116 is determines ifperformance and/or context data is known for the new feature of thecluster. For example, the algorithm corresponding to the feature vectormay include documentation that identifies performance data (e.g., theheat memory, speed, resources required, amount of memory, cache, orstack used, security, etc.) and/or context data (e.g., in whatcircumstances, technology, etc. is the function best used in conjunctionwith). If the performance and/or context data is known (block 216: YES),control continues to block 226. If some or all of the performance and/orcontext data is not known (block 216: NO), the example recommender MLM116 determines if some and/or all of the performance and/or context datacan be determined analytically (e.g., by processing the code itselfwithout actually executing the code) (block 218).

If the example recommender MLM 116 determines that some and/or all ofthe performance and/or context data cannot be determined analytically(block 218: NO), control continues to block 222. If the examplerecommender MLM 116 determines that some and/or all of the performanceand/or context data can be determined analytically (block 218: YES), theexample recommender MLM 116 determines the performance/contextual dataanalytically by processing the feature vector (block 220). At block 222,the example recommender MLM 116 determines if any performance and/orcontext data is still missing.

If the example recommender MLM 116 determines if some performance and/orcontext data is not still missing (block 222: NO), control continues toblock 226. If the example recommender MLM 116 determines if someperformance and/or context data is still missing (block 222: YES), theexample recommender MLM 116 instructs the function executor 120 toexecute the algorithm corresponding to the feature vector and theexample recommender MLM 116 and/or the example function executor 120determines the performance and/or context data based on the execution ofthe algorithm (block 224).

At block 226, the example recommender MLM 116 updates the similaritymatrix (or other type of recommendation organization) for the updatedcluster(s) based on the performance and/or contextual data of the newlyadded feature vector. For example, if the similarity matrix ranksalgorithm corresponding to the feature vectors of the cluster accordingto particular performance metrics and/or contextual metrics, the examplerecommender MLM 116 adjusts the rankings based on the performance and/orcontextual data of the new algorithm. At block 228, the examplerecommender MLM 116 stores the updated similarity matrix in the examplestorage 118. In this manner, when new code is obtained, the examplerecommender MLM 116 can make recommendations to the user for parts ofthe new code based on the updated similarity matrix.

FIGS. 3A-3B illustrate an example flowchart 300 representative ofexample machine readable instructions which may be executed to implementthe example computing system 100 of FIG. 1 to provide machine assistantprogramming to a user.

At block 301, the example feature extractor 108 obtains the example newcode 110. As described above in conjunction with FIG. 1 , the new code110 may be code generated by a user that has been successfully compiledwithout errors. At block 302, the example feature extractor 108determine is if the new code 110 includes section(s) that have beentagged by a user for recommendation. For example, before or aftersuccessful compilation of code, the user may tag a section of the codefor recommendations. In this manner, the example machine programmingsolver 112 can provide recommendations corresponding to the taggedsection of code.

If the example new code 110 does not include sections(s) tagged forrecommendation (block 302: NO), control continues to block 306. If theexample new code 110 includes sections(s) tagged for recommendation(block 302: YES), the example feature extractor 108 breaks the taggedsection(s) into block(s) of code (block 304). At block 306, the examplefeature extractor 108 automatically breaks different algorithms of thenew code 110 into block(s) of code. The algorithms could be the entirecode, a section of the code (e.g., a part of the entire code, a class, amethod, etc.), a subsection of the code (e.g., a function within thecode and/or one or more lines of code within the entire code, the class,the method, etc.), etc. At block 308, the example feature extractor 108selects a first block of the block(s) of code. At block 310, the examplefeature extractor 108 converts the selected block into a feature vector(e.g., representative of the selected block of code that can beprocessed by the example machine programming solver 112).

At block 312, the example cluster MLM 114 determines the similaritybetween the feature vector and the clusters developed during training.For example the cluster MLM 114 may determine a similarity using aEuclidean distance, a cosine similarity, etc. At block 314, the examplecluster MLM 114 determines if the feature vector corresponds to acluster. For example, if the example cluster MLM 114 determines that thefeature vector is similar to one of the clusters (e.g., based on athreshold amount of similarity), the cluster MLM 114 determines that thefeature vector corresponds to the cluster. If the example cluster MLM114 determines that the feature vector is not similar to any of theclusters, the example cluster MLM 114 determines that the feature vectordoes not correspond to a cluster. If the example cluster MLM 114determines that the feature vector does not correspond to a cluster(block 314: NO), the example cluster MLM 114 transmits the featurevector to the training database 102 of FIG. 1 (block 316). In thismanner, the feature can be used to train the example machine programmingsolver 112 at a later point in time. In some examples, the examplecluster MLM 114 generates a new cluster when the feature vector does notcorrespond to a cluster.

If the example cluster MLM 114 determines that the feature correspondsto a cluster (block 314: YES), the example cluster MLM 114 identifiesthe cluster that corresponds to the feature vector (e.g., the clusterthat is most similar to the feature vector) (block 318). At block 320,the example cluster MLM 114 passes the cluster to the examplerecommender MLM 116. At block 322, the example recommender MLM 116determines if the parameters to optimize are known. For example, theuser may have provided one or more parameters (e.g., speed, efficiency,memory consumption, security, robustness, etc.) that the user desires tooptimize. If the example recommender MLM 116 determines if theparameters to optimize are not known (block 322: NO), control continuesto block 326. If the example recommender MLM 116 determines if theparameters to optimize are known (block 322: YES), the examplerecommender MLM 116 ranks options (e.g., feature vectors correspondingto algorithms stored in the example storage 118) from the clustercorresponding to the feature vector based on the parameter(s) to beoptimize (block 324). For example, if the parameters to be optimize arespeed and security, the example recommender MLM 116 ranks algorithms inthe cluster based on speed, security, and/or a combination of speed andsecurity.

At block 326, the example recommender MLM 116 determines if contextinformation is known. The context information may be related to thepurpose of the new code 110, the technology used with the new codes 110,and/or the availability of processor resources, memory, and/or any othercurrent characteristic(s) of the example computing system 100. Forexample, the recommender MLM 116 may process the new code 110 to look atthe semantics of what the example is trying to do with respect to aparticular technology and/or may look at the current availability ofresources to determine if a particular algorithm more appropriatelyperforms the same or a substantially similar function to one or morealgorithms of the new code 110. If the example recommender MLM 116determines that context information is not known (block 326: NO),control continues to block 330 of FIG. 3B. If the example recommenderMLM 116 determines that context information is known (block 326: YES),the example recommender MLM 116 adjusts the rankings for block based onthe context information (block 328). For example, the recommender MLM116 may more heavily weigh feature vectors of the cluster that areoptimized and/or correspond more closely to the context information.

At block 330 of FIG. 3B, the example feature extractor 108 determines ifthere is an additional block of the new code 110 to analyze (e.g.,process to convert into a feature vector). In some examples, the featureextractor 108 converts all block(s) of code at one point in time andtransmits the vectors (e.g., separately or together as a matrix) to theexample machine programming solver 112 for processing. If the examplefeature extractor 108 determines that there are additional block(s) ofcode to analyze (block 330: YES), the example feature extractor 108selects one of the subsequent block(s) of code (block 332) and controlreturns to block 310 of FIG. 3A. If the example feature extractor 108determines that there are not additional block(s) of code to analyze(block 330: NO), the example recommender MLM 116 instructs a userinterface to present the rankings for the blocks of code correspondinggot he ranked feature vectors as recommendations to the user (block334). In some examples, another device may convert the ranked featuresinto the blocks of code to display to the user.

In some examples, the ranking correspond to the code that satisfy theparameters to optimize and/or the context information. For example, ifthe user has selected speed to be optimized and the contextualinformation identifies that the availability of memory is low and thatthere are alternatives that are more efficient with respect to what theuser is attempting to do (e.g., a particular communication protocol),the example recommender MLM 116 may display alternative options for partof the new code 110 that is fastest, most memory efficient, and/or moreefficient with respect to what the user is attempting to do.Additionally, the example recommender MLM 116 may identify X or morealternative options that satisfy one or more of the parameters to beoptimized and/or the context information.

At block 336, the example recommender MLM 116 determines if the userused any of the presented recommendation to replace part of the new code110. If the example recommender MLM 116 determines that the user has notused any of the presented recommendation to replace part of the new code110 (block 336: NO), control ends. If the example recommender MLM 116determines that the user has used any of the presented recommendation toreplace part of the new code 110 (block 336: YES), the examplerecommender MLM 116 updates the weight(s) and/or rankings of the storedsimilarity matrix (or other recommendation organization) for the featurevector(s) of a cluster corresponding to the user selectedrecommendation(s) (block 338).

FIG. 4 is a block diagram of an example processor platform 400structured to execute the instructions of FIGS. 2, 3A, and/or 3Btoimplement the example computing device 100 of FIG. 1 . The processorplatform 400 can be, for example, a server, a personal computer, aworkstation, a self-learning machine (e.g., a neural network), a mobiledevice (e.g., a cell phone, a smart phone, a tablet such as an iPad™), apersonal digital assistant (PDA), an Internet appliance, or any othertype of computing device.

The processor platform 400 of the illustrated example includes aprocessor 412. The processor 412 of the illustrated example is hardware.For example, the processor 412 can be implemented by one or moreintegrated circuits, logic circuits, microprocessors, GPUs, DSPs, orcontrollers from any desired family or manufacturer. The hardwareprocessor may be a semiconductor based (e.g., silicon based) device. Inthis example, the processor the example feature extractor 108, theexample cluster MLM 114, the example recommender MLM 116, and theexample function executor 120 of FIG. 1 .

The processor 412 of the illustrated example includes a local memory 413(e.g., a cache). The processor 412 of the illustrated example is incommunication with a main memory including a volatile memory 414 and anon-volatile memory 416 via a bus 418. The volatile memory 414 may beimplemented by Synchronous Dynamic Random Access Memory (SDRAM), DynamicRandom Access Memory (DRAM), RAMBUS® Dynamic Random Access Memory(RDRAM®) and/or any other type of random access memory device. Thenon-volatile memory 416 may be implemented by flash memory and/or anyother desired type of memory device. Access to the main memory 414, 416is controlled by a memory controller.

The processor platform 400 of the illustrated example also includes aninterface circuit 420. The interface circuit 420 may be implemented byany type of interface standard, such as an Ethernet interface, auniversal serial bus (USB), a Bluetooth® interface, a near fieldcommunication (NFC) interface, and/or a PCI express interface.

In the illustrated example, one or more input devices 422 are connectedto the interface circuit 420. The input device(s) 422 permit(s) a userto enter data and/or commands into the processor 412. The inputdevice(s) can be implemented by, for example, an audio sensor, amicrophone, a camera (still or video), a keyboard, a button, a mouse, atouchscreen, a track-pad, a trackball, isopoint and/or a voicerecognition system.

One or more output devices 424 are also connected to the interfacecircuit 420 of the illustrated example. The output devices 424 can beimplemented, for example, by display devices (e.g., a light emittingdiode (LED), an organic light emitting diode (OLED), a liquid crystaldisplay (LCD), a cathode ray tube display (CRT), an in-place switching(IPS) display, a touchscreen, etc.), a tactile output device, and/orspeaker. The interface circuit 420 of the illustrated example, thus,typically includes a graphics driver card, a graphics driver chip and/ora graphics driver processor.

The interface circuit 420 of the illustrated example also includes acommunication device such as a transmitter, a receiver, a transceiver, amodem, a residential gateway, a wireless access point, and/or a networkinterface to facilitate exchange of data with external machines (e.g.,computing devices of any kind) via a network 426. The communication canbe via, for example, an Ethernet connection, a digital subscriber line(DSL) connection, a telephone line connection, a coaxial cable system, asatellite system, a line-of-site wireless system, a cellular telephonesystem, etc.

The processor platform 400 of the illustrated example also includes oneor more mass storage devices 428 for storing software and/or data.Examples of such mass storage devices 428 include floppy disk drives,hard drive disks, compact disk drives, Blu-ray disk drives, redundantarray of independent disks (RAID) systems, and digital versatile disk(DVD) drives.

The machine executable instructions 432 of FIGS. 2, 3A, and/or 3B may bestored in the mass storage device 428, in the volatile memory 414, inthe non-volatile memory 416, and/or on a removable non-transitorycomputer readable storage medium such as a CD or DVD.

From the foregoing, it will be appreciated that example methods,apparatus and articles of manufacture have been disclosed that providemachine assisted programming. During training, examples disclosed hereingenerate clusters of similar features that correspond to best methodsfor the features with respect to one or more parameters and/or context.In this manner, when a user generates new code, examples disclosedherein can break the code into blocks of code and determine if there isa cluster that corresponds to one or more of the blocks of code. If acluster is available, examples disclosed herein provide recommendationsof other type of code that can replace the new code to optimize one ormore particular parameters. Additionally, examples disclosed herein,during implementation of machine learning models, create more efficientcode that results in more efficient use of computer resources (e.g.,faster, less resources used, more secure operation, less memory use,etc.). The disclosed methods, apparatus and articles of manufacture areaccordingly directed to one or more improvement(s) in the functioning ofa computer.

Example methods, apparatus, systems, and articles of manufacture toprovide machine assisted programming are disclosed herein. Furtherexamples and combinations thereof include the following: Example 1includes an apparatus to provide machine assisted programming, theapparatus including a feature extractor to convert compiled code into afirst feature vector, a first machine leaning model to identify acluster of stored feature vectors corresponding to the first featurevector, and a second machine learning model to recommend a secondalgorithm corresponding to a second feature vector of the cluster basedon a comparison of a parameter of a first algorithm corresponding to thefirst feature vector and the parameter of the second algorithm.

Example 2 includes the apparatus of example 1, wherein the first machinelearning model is to identify the cluster based on a similarity betweenthe first feature vector and the stored feature vectors of the cluster.

Example 3 includes the apparatus of example 1, wherein the first machinelearning model is to generate a new cluster for the first feature vectorwhen there is no clusters that are similar to the first feature vector.

Example 4 includes the apparatus of example 1, wherein the first machinelearning is trained based on at least one of local libraries, externallibraries, internal code, or new code that does not match a cluster.

Example 5 includes the apparatus of example 1, wherein the secondmachine learning model is to compare efficiency of parameters foralgorithms corresponding to stored feature vectors within the cluster.

Example 6 includes the apparatus of example 5, wherein the secondmachine learning model is to determine the efficiency of the parametersfor algorithms corresponding to the stored feature vectors based ondocumentation corresponding to the algorithms.

Example 7 includes the apparatus of example 5, further including afunction executor to execute a third algorithm corresponding to a storedfeature, the second machine learning model to determine the efficiencyof the parameter for the algorithm based on the execution of the thirdalgorithm.

Example 8 includes the apparatus of example 1, wherein the parameter isselected by a user.

Example 9 includes the apparatus of example 1, wherein the secondmachine learning model is determined based on context informationincluding at least one of availability of resources or purpose of thecompiled code.

Example 10 includes the apparatus of example 1, wherein therecommendation is displayed to a user.

Example 11 includes the apparatus of example 1, wherein the secondmachine learning model is to update recommendations based on userfeedback to the recommendation.

Example 12 includes a non-transitory computer readable storage mediumcomprising instructions which, when executed, cause a machine to atleast convert compiled code into a first feature vector, identify acluster of stored feature vectors corresponding to the first featurevector, and recommend a second algorithm corresponding to a secondfeature vector of the cluster based on a comparison of a parameter of afirst algorithm corresponding to the first feature vector and theparameter of the second algorithm.

Example 13 includes the computer readable storage medium of example 12,wherein the instructions cause the machine to identify the cluster basedon a similarity between the first feature vector and the stored featurevectors of the cluster.

Example 14 includes the computer readable storage medium of example 12,wherein the instructions cause the machine to generate a new cluster forthe first feature vector when there is no clusters that are similar tothe first feature vector.

Example 15 includes the computer readable storage medium of example 12,wherein the instructions cause the machine to a first machine learningis trained based on at least one of local libraries, external libraries,internal code, or new code that does not match a cluster.

Example 16 includes the computer readable storage medium of example 12,wherein the instructions cause the machine to compare efficiency ofparameters for algorithms corresponding to stored feature vectors withinthe cluster.

Example 17 includes the computer readable storage medium of example 16,wherein the instructions cause the machine to determine the efficiencyof the parameters for algorithms corresponding to the stored featurevectors based on documentation corresponding to the algorithms.

Example 18 includes the computer readable storage medium of example 16,wherein the instructions cause the machine to execute a third algorithmcorresponding to a stored feature, the second machine learning model todetermine the efficiency of the parameter for the algorithm based on theexecution of the third algorithm.

Example 19 includes the computer readable storage medium of example 12,wherein the parameter is selected by a user.

Example 20 includes the computer readable storage medium of example 12,wherein the instructions cause the machine to determine a machinelearning model based on context information including at least one ofavailability of resources or purpose of the compiled code.

Example 21 includes the computer readable storage medium of example 12,wherein the recommendation is displayed to a user.

Example 22 includes the computer readable storage medium of example 12,wherein the instructions cause the machine to update recommendationsbased on user feedback to the recommendation.

Example 23 includes a method to provide machine assisted programming,the method including converting compiled code into a first featurevector, identifying, using a first machine learning model, a cluster ofstored feature vectors corresponding to the first feature vector, andrecommending, using a second machine learning model, a second algorithmcorresponding to a second feature vector of the cluster based on acomparison of a parameter of a first algorithm corresponding to thefirst feature vector and the parameter of the second algorithm.

Example 24 includes the method of example 23, further includingidentifying the cluster based on a similarity between the first featurevector and the stored feature vectors of the cluster.

Example 25 includes the method of example 23, further includinggenerating a new cluster for the first feature vector when there is noclusters that are similar to the first feature vector.

Example 26 includes the method of example 23, further including trainingthe first machine learning model based on at least one of locallibraries, external libraries, internal code, or new code that does notmatch a cluster.

Example 27 includes the method of example 23, further includingcomparing efficiency of parameters for algorithms corresponding tostored feature vectors within the cluster.

Example 28 includes the method of example 27, further includingdetermining the efficiency of the parameters for algorithmscorresponding to the stored feature vectors based on documentationcorresponding to the algorithms.

Example 29 includes the method of example 27, further includingexecuting a third algorithm corresponding to a stored feature, thesecond machine learning model to determine the efficiency of theparameter for the algorithm based on the execution of the thirdalgorithm.

Example 30 includes the method of example 23, wherein the parameter isselected by a user.

Example 31 includes the method of example 23, further including trainingthe second machine learning model based on context information includingat least one of availability of resources or purpose of the compiledcode.

Example 32 includes the method of example 23, wherein the recommendationis displayed to a user.

Example 33 includes the method of example 23, further including updatingrecommendations based on user feedback to the recommendation. Althoughcertain example methods, apparatus and articles of manufacture have beendisclosed herein, the scope of coverage of this patent is not limitedthereto. On the contrary, this patent covers all methods, apparatus andarticles of manufacture fairly falling within the scope of the claims ofthis patent.

The following claims are hereby incorporated into this DetailedDescription by this reference, with each claim standing on its own as aseparate embodiment of the present disclosure.

What is claimed is:
 1. A non-transitory computer readable storage mediumcomprising instructions to cause processor circuitry to at least:execute a machine learning model to generate a first code recommendationfor programming code, the first code recommendation being associatedwith security of the programming code; cause output of the first coderecommendation via a user interface; update the machine learning modelbased on feedback obtained via the user interface; determine aperformance of the programming code; generate a second coderecommendation, the second code recommendation being associated with theperformance of the programming code; and cause output of the second coderecommendation via the user interface.
 2. The non-transitory computerreadable storage medium of claim 1, wherein the performance of theprogramming code corresponds to at least one of an amount of processorresources to execute the programming code, an efficiency of theprogramming code, or a latency associated with the programming code. 3.The non-transitory computer readable storage medium of claim 1, whereinthe instructions cause the one or more processors to convert theprogramming code into one or more feature vectors associated with theprogramming code, the machine learning model to generate the first coderecommendation based on the one or more feature vectors.
 4. Thenon-transitory computer readable storage medium of claim 1, wherein theinstructions cause the one or more processors to cause code to beinserted into the programming code in response to user selection of thefirst code recommendation.
 5. The non-transitory computer readablestorage medium of claim 1, wherein a user provides the feedback via theuser interface.
 6. The non-transitory computer readable storage mediumof claim 1, wherein the feedback corresponds to user selection of thefirst code recommendation.
 7. An apparatus comprising: memory; computerreadable instructions; and processor circuitry to execute the computerreadable instructions to: execute a machine learning model to generate afirst code recommendation for programming code, the first coderecommendation being associated with security of the programming code;cause output of the first code recommendation via a user interface;update the machine learning model based on feedback obtained via theuser interface; determine a performance of the programming code;generate a second code recommendation, the second code recommendationbeing associated with the performance of the programming code; and causeoutput of the second code recommendation via the user interface.
 8. Theapparatus of claim 7, wherein the performance of the programming codecorresponds to at least one of an amount of processor resources toexecute the programming code, an efficiency of the programming code, ora latency associated with the programming code.
 9. The apparatus ofclaim 7, wherein the processor circuitry is to convert the programmingcode into one or more feature vectors associated with the programmingcode, the machine learning model to generate the first coderecommendation based on the one or more feature vectors.
 10. Theapparatus of claim 7, wherein the processor circuitry is to cause codeto be inserted into the programming code in response to user selectionof the first code recommendation.
 11. The apparatus of claim 7, whereina user provides the feedback via the user interface.
 12. The apparatusof claim 7, wherein the feedback corresponds to user selection of thefirst code recommendation.
 13. A method comprising: executing a machinelearning model to generate a first code recommendation for programmingcode, the first code recommendation being associated with security ofthe programming code; outputting of the first code recommendation via auser interface; updating the machine learning model based on feedbackobtained via the user interface; determining a performance of theprogramming code; generating a second code recommendation, the secondcode recommendation being associated with the performance of theprogramming code; and outputting of the second code recommendation viathe user interface.
 14. The method of claim 13, wherein the performanceof the programming code corresponds to at least one of an amount ofprocessor resources to execute the programming code, an efficiency ofthe programming code, or a latency associated with the programming code.15. The method of claim 13, further including converting the programmingcode into one or more feature vectors associated with the programmingcode, the machine learning model to generate the first coderecommendation based on the one or more feature vectors.
 16. The methodof claim 13, further including causing code to be inserted into theprogramming code in response to user selection of the first coderecommendation.
 17. The method of claim 13, wherein a user provides thefeedback via the user interface.
 18. The method of claim 13, wherein thefeedback corresponds to user selection of the first code recommendation.